22 research outputs found
Information Theoretic Limits for Standard and One-Bit Compressed Sensing with Graph-Structured Sparsity
In this paper, we analyze the information theoretic lower bound on the
necessary number of samples needed for recovering a sparse signal under
different compressed sensing settings. We focus on the weighted graph model, a
model-based framework proposed by Hegde et al. (2015), for standard compressed
sensing as well as for one-bit compressed sensing. We study both the noisy and
noiseless regimes. Our analysis is general in the sense that it applies to any
algorithm used to recover the signal. We carefully construct restricted
ensembles for different settings and then apply Fano's inequality to establish
the lower bound on the necessary number of samples. Furthermore, we show that
our bound is tight for one-bit compressed sensing, while for standard
compressed sensing, our bound is tight up to a logarithmic factor of the number
of non-zero entries in the signal
Provable Sample Complexity Guarantees for Learning of Continuous-Action Graphical Games with Nonparametric Utilities
In this paper, we study the problem of learning the exact structure of
continuous-action games with non-parametric utility functions. We propose an
regularized method which encourages sparsity of the coefficients of
the Fourier transform of the recovered utilities. Our method works by accessing
very few Nash equilibria and their noisy utilities. Under certain technical
conditions, our method also recovers the exact structure of these utility
functions, and thus, the exact structure of the game. Furthermore, our method
only needs a logarithmic number of samples in terms of the number of players
and runs in polynomial time. We follow the primal-dual witness framework to
provide provable theoretical guarantees.Comment: arXiv admin note: text overlap with arXiv:1911.0422
Outlier-robust Estimation of a Sparse Linear Model Using Invexity
In this paper, we study problem of estimating a sparse regression vector with
correct support in the presence of outlier samples. The inconsistency of
lasso-type methods is well known in this scenario. We propose a combinatorial
version of outlier-robust lasso which also identifies clean samples.
Subsequently, we use these clean samples to make a good estimation. We also
provide a novel invex relaxation for the combinatorial problem and provide
provable theoretical guarantees for this relaxation. Finally, we conduct
experiments to validate our theory and compare our results against standard
lasso
Fair Sparse Regression with Clustering: An Invex Relaxation for a Combinatorial Problem
In this paper, we study the problem of fair sparse regression on a biased
dataset where bias depends upon a hidden binary attribute. The presence of a
hidden attribute adds an extra layer of complexity to the problem by combining
sparse regression and clustering with unknown binary labels. The corresponding
optimization problem is combinatorial, but we propose a novel relaxation of it
as an \emph{invex} optimization problem. To the best of our knowledge, this is
the first invex relaxation for a combinatorial problem. We show that the
inclusion of the debiasing/fairness constraint in our model has no adverse
effect on the performance. Rather, it enables the recovery of the hidden
attribute. The support of our recovered regression parameter vector matches
exactly with the true parameter vector. Moreover, we simultaneously solve the
clustering problem by recovering the exact value of the hidden attribute for
each sample. Our method uses carefully constructed primal dual witnesses to
provide theoretical guarantees for the combinatorial problem. To that end, we
show that the sample complexity of our method is logarithmic in terms of the
dimension of the regression parameter vector
A Simple Unified Framework for High Dimensional Bandit Problems
Stochastic high dimensional bandit problems with low dimensional structures
are useful in different applications such as online advertising and drug
discovery. In this work, we propose a simple unified algorithm for such
problems and present a general analysis framework for the regret upper bound of
our algorithm. We show that under some mild unified assumptions, our algorithm
can be applied to different high dimensional bandit problems. Our framework
utilizes the low dimensional structure to guide the parameter estimation in the
problem, therefore our algorithm achieves the best regret bounds in the LASSO
bandit, as well as novel bounds in the low-rank matrix bandit, the group sparse
matrix bandit, and in a new problem: the multi-agent LASSO bandit
Invex Programs: First Order Algorithms and Their Convergence
Invex programs are a special kind of non-convex problems which attain global
minima at every stationary point. While classical first-order gradient descent
methods can solve them, they converge very slowly. In this paper, we propose
new first-order algorithms to solve the general class of invex problems. We
identify sufficient conditions for convergence of our algorithms and provide
rates of convergence. Furthermore, we go beyond unconstrained problems and
provide a novel projected gradient method for constrained invex programs with
convergence rate guarantees. We compare and contrast our results with existing
first-order algorithms for a variety of unconstrained and constrained invex
problems. To the best of our knowledge, our proposed algorithm is the first
algorithm to solve constrained invex programs